Advances in network technology, like peer-to-peer networks on the Internet, sensor networks, etc., have highlighted the need for efficient ways to deal with large amounts of data distributed over a set of nodes. For example, in a peer-to-peer system, a user may need to learn about the global distribution of resources for indexing or load balancing. In another example, in a distributed agent system, each agent may need some global information of the environment through collaborative learning for decision making.
Some systems use gossip-based procedures based on parametric models, such as Gaussian mixture models, for data computation and distributed density estimation. In a gossip-based procedure or protocol, each node or node device in a system repeatedly contacts other nodes at random to exchange information. Gossip-based parametric models may not always be suitable for distributed learning, and are highly sensitive to initial parameters.